Deep Q-Learning With Q-Matrix Transfer Learning for Novel Fire Evacuation Environment

Deep reinforcement learning (RL) is achieving significant success in various applications like control, robotics, games, resource management, and scheduling. However, the important problem of emergency evacuation, which clearly could benefit from RL, has been largely unaddressed. Indeed, evacuation a complex task that difficult to solve with RL. An situation highly dynamic, lot changing variables constraints make it challenging solve. Also, there no standard benchmark environment available can be used train RL agents for evacuation. A realistic design. In this article, we propose first fire planning. The modeled as graph capturing building structure. It consists features spread, uncertainty, bottlenecks. implementation our OpenAI gym format, facilitate future research. We also new approach entails pretraining network weights DQN-based agent [DQN/Double-DQN (DDQN)/Dueling-DQN] incorporate information on shortest path exit. achieved by using tabular $Q$ -learning learn model’s graph. This transferred deliberately overfitting -matrix. Then, pretrained DQN model trained generate optimal under time varying conditions due bottlenecks, uncertainty. perform comparisons proposed state-of-the-art algorithms DQN, DDQN, Dueling-DQN, PPO, VPG, state-action-reward-state-action (SARSA), actor–critic method, ACKTR. results show method able outperform models huge margin including original models. Finally, tested large real consisting 91 rooms, possibility move any other room, hence giving 8281 actions. order reduce action space, strategy involves one step simulation. That is, an importance vector added final output acts attention mechanism. Using strategy, space reduced 90.1%. manner, deal spaces. Hence, achieves near performance world environment.